When The Safe Option Goes Wrong